On Optimal Pairwise Linear Classi ers for Normal Distributions: The Two-Dimensional Case

Computing linear classi ers is a very important problem in statistical Pattern Recognition (PR). These classi ers have been investigated by the PR community extensively since they are the ones which are both easy to implement and comprehend. It is well known that when dealing with normally distributed classes, the optimal discriminant function for two-classes is linear only when the covariance matrices are equal. Other approaches, such as the Fisher's discriminant, the perceptron algorithm, minimum square distance classi ers, etc., have solved this problem by generating a linear classi er in normal and non-normal distributions, but these classi ers are typically suboptimal. In this paper we shall focus on some special cases of the normal distribution with non-equal covariance matrices. We present a complete analysis of the case when the classi er is pairwise linear, and to our knowledge this is a pioneering work for the use of such classi ers in any are of statistical PR. We shall determine the conditions that the mean vectors and covariance matrices have to satisfy in order to obtain the optimal linear classi er. However, as opposed to the state of the art, in all the cases discussed here, the linear classi er is given by a pair of straight lines, which is a particular case of the general equation of second degree. One of these cases is when we have two overlapping classes with equal means, which is a general case of the Minsky's Paradox for the Perceptron. We present a general linear classi er for this particular case which can be obtained directly from the parameters of the distribution. Numerous other analytic results for two dimensional normal random vectors have been derived. Finally, we have also provided some empirical results in all the cases, and demonstrated that these linear classi ers achieve very good performance.